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Humans differ from other animals most remarkably in 
their ability to learn. It is clear that although throughout the 
history of mankind technological learning has taken place, 
although until three or four hundred years ago change oc- 
curred very slowly. One reason for this was that in order to 
learn something - for example, how to make fire or cham- 
pagne - two rare events needed to coincide: (a) an informa- 
tive event had to occur , and (b) a person able to draw logical 
conclusion and to act on them had to be aware of that 
informative event. 

Passive surveillance is a way of increasing the probability 
that the rare informative event will be constructively taken 
note of and is exemplified by quality charting methods. 
Thus a Shewhart chart is a means to ensure that possibly 
informative events are brought to the attention of those who 
may be able to discover in them an "assignable cause" [l] 1 
and act appropriately. 
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and National Science Foundation Grant DMS-8420968. 

■Figures in brackets indicace literature references. 



Active intervention by experimentation aims, in addition, 
to increase the probability of an informative event actually 
occurring. A designed experiment conducted by a qualified 
experimenter can dramatically increase the probability of 
learning because it increases simultaneously the probability 
of an informative event occurring and also the probability of 
the event being constructively witnessed. Recently there has 
been much use of experimental design in Japanese industry 
particularly by Genichi Taguchi [2] and his followers. In 
off-line experimentation he has in particular emphasized the 
use of highly fractionated designs and orthogonal arrays and 
the minimization of variance. 

In the remainder of this paper we briefly outline some 
recent research on the use of such screening designs. 



1. Use of Screening Designs to 
Identify "Active" Factoring 

Table 1 shows in summary a highly fractionated two-level 
factorial design employed as a screening design in an off- 
line welding experiment performed by the National Railway 
Corporation of Japan [2J. In the column to the right of the 
table is shown the observed tensile strength of the weld, one 
of several quality characteristics measured. 
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The design was chosen on Ihe assumption that in addition 
to main effects only the two-factor interactions AC, AG, 
AH, and GH were expected to be present. On that supposi- 
tion, al! nine main effects and the four selected two-factor 
interactions can be separately estimated by appropriate or- 
thogonal contrasts, the two remaining contrasts correspond- 
ing to the columns labelled e] and e 2 measure only experi- 
mental error. Below the table are shown the grand average, 
the 15 effect contrasts, and the effects pJotted on a dot 
diagram. The effects plotted on normal probability paper 
suggested that, over the ranges studied, only factors B and 
C affect tensile location by amounts not readily attributed to 
noise. 

If this conjecture is true, then, at least appoximately, the 
16 runs could be regarded as four replications of a 2 2 facto- 
rial design in factors B and C only. However, when the 
results are plotted in figure 1 so as to reflect this, inspection 
suggests the existence of a dramatic effect of a different 
kind — when factor C is at its plus level the spread of the data 
appears much larger 3 than when it is at its minus level. 
Thus, in addition to detecting shift in location due to B and 
C, the experiment may also have detected what we will call 
a dispersion effect due to C. The example raises the general 
possibility of analyzing unreplicated designs for dispersion 
effects as well as for the more usual location effects. 



2 To facilitate later discussion, we have set out the design and labelled the levels 
somewhat differently from |2] . 

-'Data of this kind might be accounted for by (he effect of one or more variables 
other than B that affected tensile strength only at Ihe "plus level" of C (only when the 
alternative material was used). Analysis of the eight runs made at the plus level of C 
does not support this possibility, however. 



2. Rationales for Using 
Screening Designs 

Before proceeding we need to consider the question, "In 
what situations are screening designs, such as highly frac- 
tionated factorials, useful?" 

2.1. Effect Sparsity. A common industrial problem is to 
find from a rather large number of factors those few that are 
responsible for large effects. The idea is comparable to that 
which motivates the use in quality control studies of the 
"Pareto diagram." (See, for example, [3]). The situation is 
approximated by postulating that only a small proportion of 
effects will be "active" and the rest "inert". We call this the 
postulate of effect sparsity. For studying such situations, 
highly fractionated designs and other orthogonal arrays 
[2,4,5,6] which can screen moderately large numbers of 
variables in rather few runs are of great interest. Two main 
rationalizations have been suggested for the use of these 
designs; both ideas rely on the postulate of effect sparsity 
but in somewhat different ways. 

2.2. Rationale Based on Prior Selection of Important 
Interactions. It is argued (see for example [7]) that in some 
circumstances physical knowledge of the process will make 
only a few interactions likely and that the remainder may be 
assumed negligible. For example, in the welding experi- 
ment described above there were 36 possible two-factor 
interactions between the nine factors, but only four were 
regarded as likely, leaving 32 such interactions assumed 
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Figure 1— Tensile strength data as 
four replicates of a 2 2 fac- 
torial design in factors B 
and C only. 



40 42 44 46 



Drying Period B 



+ 



496 



negligible. The difficulty with this idea is that in many 
applications the picking out of a few "likely" interactions is 
difficult if not impossible. Indeed the investigator might 
justifiably protest that, in the circumstance where an exper- 
iment is needed to determine which first order (main) effects 
are important, it is illogical that he be expected to guess in 
advance which effects of second order (interactions) are 
important. 

2.3. Projective Rationale Factor Sparsity. A somewhat 
different notion is that of factor sparsity. Thus suppose that, 
of the k factors considered, only a small subset of vaguely 
known size J, whose identity is, however, unknown , will be 
active in providing main effects and interactions within the 
subset. Arguing as in [8], a two-level design enabling us to 
study such a system is a fraction of resolution R =</ + 1 (or 
in the terminology of 16), an array of strength d) which 
produces complete factorials (possibly replicated) in every 
one of the (*j) spaces of d=R-{ dimensions. For example, 
we have seen that on the assumption that only factors B and 
C are important, the welding design could be regarded as 
four replicates of a 2 2 factorial in just those two factors. But 
because the design is of resolution R=3 the same would 
have been true for any of the 36 choices of two out of the 
nine factors tested. Thus the design would be appropriate if 
it were believed that not more than two of the factors were 
likely to be "active". 



For further illustration we consider again the !6-run or- 
thogonal array of table 1 and, adopting a Toman subscript to 
denote the resolutions of the design, we indicate in table 2 
various ways in which that array might be used. It may be 
shown that 

(a) If we associated the 1 5 contrast columns of the design 
with 15 factors, we would generate a 2}^' il design providing 
two-fold replication of 2 2 factorials in every one of the 105 
two-dimensional projections. 

(b) If we associated only columns 1 . 2, 4, 7, 8, II, 13, 
and 14 with eight factors we would generate a 2f v 4 design 
providing two-fold replication of 2 J factorials in every one 
of the 56 three-dimensional projections. 

(c) If we associated only columns 1 , 2, 4. 8. and 15 with 
five factors we would generate a 2 V ' ( design providing a 2 4 
factorial in every one of the four-dimensional projections. 

(d) If we associated only columns 1,2,4. ami & with four 
factors we would obtain the complete 2 4 design from which 
this orthogonal array was in fact generated. 

Designs (a), (b), and (c) would thus be appropriate for 
situations where we believed respectively that not more than 
2. 3. or 4 factors would be active 4 . Notice that intermediate 



4 The designs give partial coverage for a larger number of factors, for 
example <[XI ( 1961 >) 5f> til ihe 70 four-diniensonal projections of the 2f^ 
vield a full factorial in four variables. 



Table 1. A fractional two-level design used in a welding experiment showing observed tensile strength and effects. 
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Table 2. Some alternative uses of the orthogonal array. 
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values of k could be accommodated by suitably omitting 
certain columns. Thus the welding design is a 2Jif arrange- 
ment which can be obtained by omitting six columns from 
the complete 2/rj' J1 . Notice Finally that for intermediate de- 
signs we can take advantage of both rationales by arranging, 
as was done for the welding design, that particular interac- 
tions are isolated. 

A discussion of the iterative model building process [9] 
characterized three steps in the iterative data analysis cycle 
indicated below 



r. 



identification > fitting diagnostic checking 



J 



Most of the present paper is concerned with model identifi- 
cation - the search for a model worthy to be formally enter- 
tained and fitted by an efficient procedure such as maximum 
likelihood. The situation we now address concerns the anal- 
ysis of fractional designs such as the welding design in the 



above context when only a few of the factors are likely to 
have effects but these may include dispersion effects as well 
as location effects. 

3. Dispersion Effects 

We again use the design of table 1 for illustration, There 
are 16 runs from which 16 quantities — the average and 15 
effect contrasts — have been calculated. Now if we were also 
interested in possible dispersion effects we could also calcu- 
late 15 variance ratios. For example, in column 1 we can 
compute the sample variance sf_ for those observations 
associated with a minus sign and compare it with the sample 
variance sf + for observations associated with a plus sign to 
provide the ratio F y =sf_ /sf + . If this is done for the welding 
data we obtain values for \nF i given in figure 2(a) J . It will 
be recalled that in the earlier analysis a large dispersion 
effect associated with factor C {column 15) was found, but 
in figure 2(a) the effect for factor C is not especially ex- 
treme, instead the dispersion effect for factor D (column 1) 
stands out from all the rest. This misleading indication oc- 
curs because we have not so far taken account of the aliasing 
of location and dispersion effects. Since 16 linearly inde- 
pendent location effects have already been calculated for the 
original data, calculated dispersion effects must be functions 



Ha this figure familiar normal theory significance levels are also shown. Obviously 
the necessary assumptions are not satisfied in this case, but these percentages provide 
& rough indication of magnitude. 
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Figure 2 — Welding experiment log dispersion effects (a) before, and (b) after elimination of location effects for B and C. 
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of these. Recently [10] a general theory of location - 

dispemon aliasing has been obtained for factorials and frac- 
tional factorials at two levels. For iLlus:ralion, in this partic- 
ular example it turns out (hat the following identity exists for 
the dispersion effect F u :hat is the ^ ratio associated with 
factor D and hence for column 1 of the design, 



Table 3. Ci'.eulEdsd effect from a 2f^ design shoeing alias structure 
ass'jmirig three factor £nd higher aider interactiojis negligible, injection 
molding e<cper.raent. 
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Now (see table 1) 14=6=2.15 and l"5=t=3. 10 are the two 
largest location effects, standing out from all the others. The 
extreme- value of F 3 associated with an apparent dispersion 
effect of factor Dil) is largely accounted for by the squared 
sum and squared difference of the location effects B and C 
which appear respectively as the last terms in the denomina- 
tor and numerator of eq (1). A natural way to proceed is to 
compute variances from the residuals obtained after elimi- 
nating large location effects . After such elimination the alias 
relations of eq (1) remain the same except that location 
effects from eliminated variables drop out. That is, zeros are 
substituted for eliminated variables. Variance analysis for 
the residuals after eliminating effects of B and C are shown 
in figure 2(b). The dispersion effect associated with C (fac- 
tor 15) is now correctly indicated as extreme. It is shown in 
foe paper referenced above how, more generally, under 
circumstances of effect sparsiry a location-dispersion model 
may be correctly identified when a few effects of both kinds 
are present. 

4. Analysis of Unreplicated 
Fractional Designs 

Another important problem in the analysis of unreplicated 
fractional designs and other orthogonal arrays concerns the 
picking out of "active" factors. A serious difficulty is that 
with unreplicated fractional designs no simple estimate of 
the experimental error variance against which to judge the 
effects is available. 

In one valuable procedure due to Cuthbert Daniel [11, 12] 
effects are plotted on normal probability paper. For illustra- 
tion table 3 shows the calculated effects from a 2& design 
used in an experiment on injection molding [13, p. 379]. 
Tliese effects aw plaited en tionnLl pmbab:li.tv paper in 
figure 3. 

An aHer-adve Bayssiam Lpproach ~14\ Ls as fo..ows: Let 
~?i^2->- ■■j"u te stan daroi zed* effects wilh 

T i = e, if effect inert 
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T s = 1.2 * a screw speed 
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s Fcn tbrte-lswl and mixed t<vo and thiee levd designs fot example , thii analysis 
is carried out after the effects are scaled so that they all hove equal -variances. 



Figure 3 — Normal plot of effects. Injection molding experiment. 
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Figure 4 — (a) Welding experiment. Posterior probability that factor i is 
active (a = 0.30, k = 10). (b) Sensitivity analysis for posterior 
probability (a = . 15 - .45, k = 5 - 15). 
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Suppose the probability that an effect is active is a. 

Let «„., be the event that a particular .set of r of the v 
factors are active, and let T (rJ be the vector of estimated 
effects corresponding to active factors of a ir) . Then, [I5J 
with p{o)<* i/o- the posterior probability that T (/) are the 
only active efFects is: 
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where S t/ .)=T ( r f) T (r , and S=T'T. In particular the marginal 
probability that an effect / is active give 7", a and k is 

proportional to 
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A study of the fractional factorials appearing in 
[7,12.13]. suggested that a might range from 0.15-0.45 
while k might range from 5 to 15. The posterior probabilities 



computed with the (roughly average) values. a=0.30 and 
k= J are shown in figure 4(a) in which N denotes the 
probability (negligible for this example) that there are no 
active effects. The results from a sensitivity analysis in 
which a and k were altered to vary over the ranges men- 
tioned above is shown in figure 4(b). 

It will be seen that figure 4(a) points to the conclusion that 
active effects are associated with columns 3, 5 and 12 of the 
design and that column 8 might possibly also be associated 
with an active factor. Figure 4(b) suggests that this conclu- 
sion is very little affected by widely different choices for a. 
and k. Further research with different choices of prior, with 
marginization with respect lo k, and with different choices 
of the distribution assumptions is being conducted. 



5. Allowance for Faulty Observations 

Recent work [ 16] has shown how a double applicalion of 
the scale-contamination model (both to the observations 
themselves as well as to the affects) can make it possible to 
allow for faulty observations in the analysis of unreplicated 
factorials or fractional factorials. 
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